Characterizing the Opportunity and Feasibility of Reconfigurable Memory Hierarchies for Improved Energy Efficiency

The end of Dennard scaling has made energy-efficiency a critical challenge in the continued increase of computing performance. An important approach to increasing energy-efficiency is hardware customization to specific application needs. In this study we explore the opportunity for memory hierarchy customization for energy-efficiency, exploring reconfigurable memory hierarchies. Using a workload of 37 diverse benchmarks, we address three key questions: 1) how much benefit is possible?, 2) how much reconfiguration is required?, and 3) can we automatically select a good memory hierarchy configuration? Our results show that the potential benefit is large – average reductions of 70% in memory hierarchy energy with no performance loss. Further, our results show that number of configurations need not be large; ten carefully chosen configurations can deliver 90% of this benefit (63% energy reduction) suggesting that configurable hierarchies may be practically realizable. Finally, we explore reuse distance as a guide to select the best memory hierarchy configuration as a first step towards automatic configuration, and show that it can effectively predict which memory hierarchies will both maintain performance and deliver energy efficiency.